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ABSTRACT 

Evaluation of freshman English programs is a process 
involving recursive steps, such as planning which elements to review, 
securing resources to carry out the review, actually reviewing the 
planned elements, and providing written reports to decision-makers. 
The four elements of an educational program— contexts, input, 
process, and product— cohere tightly, and all important elements in 
an educational program may have to be improved for significant change 
to occur, A professional evaluator of English programs plans the 
intensity of the review and is involved in deciding how to respond to 
the political-practical circumstances impinging on the review. An 
evaluator also needs substantial financial resources, a year or more 
of time, help from a committee, consultations with other 
professionals, secretarial services, and research and editorial 
assistance. Although not everyone agrees on the goals of an English 
program, an evaluator will describe the goals or contexts as they 
appear on the basis of documents, interviews, and observations, it is 
crucial that evaluators review human and non^human resources in most 
English programs. A comparison of a program's courses and sections 
with those of typical programs can be made, and if discrepancies 
exist, a formal report should suggest alternative procedures. Expert 
information on the eraluation elements of teaching-learning and 
achievements is readily available. Professional evaluators 1 insights 
can help make the process of evaluation fair, efficient, and 
thorough* (References are appended,) (NKA) 
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Professional Evaluators* Insights Applied to 
Assisting Freshman English Programs 

Evaluations are the contemporary charge* States , universities* 
and colleges express dissatisfaction with graduates* writing and then 
evaluate freshman English programs* Sometimes, as in South Dakota, 
they use a value-added model of evaluation and expect improvement in 
terms of "change scores," When scores are deemed unsatisfactory, they 
recommend improvement in goals and instruction* 

On the other hand, the chair and faculty of many departments of 
English thus evaluated frequently know that too much must be taught to 
too many underprepared freshmen through too few, often transient 
instructors* In short, resources are limited. 

Since evaluations seem to be here to stay, what can we in English 
do beyond sigh? One response is to show concern for our students by 
deepening and broadening evaluations to make them more complete and 
fair* To augment what we already know, we can turn to professional 
evaluators for insights and methods* 

Here such insights are selected from more than a dozen 
professional evaluators* These insights will be explained in the 
order in v/hich evaluators from inside or outside English departments 
might use them as parts of evaluating a freshman English program* 

Planning 

Sometimes evaluators within English conceive of evaluation as the 
simple activity of reviewing students* achievement scores* However, 
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many professional evaluators assert that evaluation is a process (PDK 
40), much as writing is a process* and involves recursive steps* 
These steps include planning which elements to review, securing 
resources to carry out the review, actually reviewing the planned 
elements, and providing written reports to those who make decisions* 

When beginning to plan, an evaluator is well advised to heed 
Stufflebeam, chairman for a Phi Delta Kappa (PDK) Committee (PDK 
218-222), and consider whether to review one, several, or all four 
elements of &n educational program* The elements, which form the CXPP 
model, are contexts, input, process, and product* "Contexts" are the 
purposes or goals of a program, Aristotle T & (12) efficient cause* 
"Input" refers to the sum of the human and non-human resources for a 
program, that is, the faculty, students, and all materials, 
Aristotle's material cause* "Process" refers to the activities of 
teachii.;, learning, and decision-making, Aristotle T s formal cause* 
"Product" refers to the freshmen's achieved skills and arts in 
communicating through English written discourse, Aristotle's final 
cause* 

As a brief history of professional evaluation shows, these 
elements of an educational program cohere tightly* Many years ago 
evaluators looked only at products* In 1932, Tyler (498-499) improved 
evaluation by studying whether the carefully stated educational 
objectives (contexts) were achieved in the students' final scores 
(product)* In the 1970's many teachers in secondary schools found a 
need to expand Tyler's approach* When large numbers of underprepared 
students entered their classrooms, they raised the slogan "No 
evaluation without remediation*" These teachers asked that neither 



4 



3 

their instruction (process) nor the students T achievements (products) 
be judged without opportunity for students to be given adequate 
preparation (input)* This history implies that all four elements of 
English papers should be evaluated* 

In addition to planning the elements to review, an evaluator of 
English programs plans the intensity of the review* A "formative*' 
evaluation, according to Scriven (40-43), is designed to improve or 
alter a program through dialogue among relevant persons while the 
program is in progress* This dialogue, termed "feedback" (PDK 
254-255) and indicated on evaluator *s models as loops on a flow chart, 
makes formative evaluation likely to win cooperation from all persons 
involved* A "summative" evaluation reviews a program to judge it 
overall and to determine whether to continue it or terminate it* 
Monitoring," a hybrid type of evaluation, employs formative 
evaluation from outside the program for the purpose of support or 
intervention* 

Planning also involves deciding how to respond to the political- 
practical circumstances impinging on the review* Professional 
evaluators, for instance, clarify their charges in writing, assess the 
possibility that their evaluations will be utilised (Rossi and Freeman 
307, 313, 320-328), and discover what constraints exist (Rossi and 
Freeman 308, 309, 313, 316-319; Worthen and Sanders 346-347)* 

Then an evaluator gathers his or her resources (PDK 165-167, 244)* 
From the outset, part of the total annual budget of a program should 
be designated for evaluation, and the sum should be substantial* 
Educational research for developing "a new curriculum or classroom 
procedure*' along with arranging the concomitant evaluation is 
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expensive; still it costs only 1% of designing and testing an 
automobile engine, and it is likely to bring "enormously more 
important" benefits, namely, "enlightened citizens" (Scriven 83). In 
addition to money, an evaluator probably needs a ye*r or more of time, 
help of a committee, consultations with a professional evaluator, 
secretarial services, a research assistant, and editorial assistance. 

Reviewing Goals or Contexts 

Then comes the actual review* Professional evaluators offer many 
alternative methods for reviewing the goals* resources, processes of 
teaching-learning, and achievements* A few methods follow* 

Not everyone agrees on the goals of freshman English* According 
to a national survey of one hundred programs (Witte, Cherry, and Meyer 
29), directors of writing programs perceive that "writing mechanically 
correct prose" is the real goal (69%) as well as the goal of the 
department (56£), university (68£), and society (72£), However, 
instructors in the programs perceive that 1f writing coherent prose" is 
the real goal (6l£) although not the goal of the department (18&), 
university (22£)» or society (23£)* "The differences suggest that any 
attempt to evaluate freshman writing programs must begin by addressing 
those different perceptions" (Witte, Cherry, and Meyer 1). 

An evaluator of an English program's goals or contexts will 
undoubtedly describe the goals as they appear on the basis of 
documents, interviews, and observations. He oc she may also assess 
these goals in a variety of ways* One way is to learn the "perceived 
needs" (Rossi and Freeman 99*106) of all those who have a right or 
"stake" in the program* These "stakeholders" include the students and 
the instructors of the courses as well as other members of the English 



department, non-English faculty members, college and university 
administrators, parents, and citizens of the state* After describing 
each group, an evaluator can assess the members T immediate and 
ultimate goals by a well-designed questionnaire sent to a stratified 
random sample* The questionnaire could use items that express levels 
of achievement in terms taken from taxonomies of cognitive and 
affective educational objectives (Bloom et al* ; Krathwohl, Bloom, and 
Masia)* After analysis is complete, if the groups disagree widely 
about needs, it is not advantageous, Hammond and Popham agree (168; 
38), to proceed with the review* Dialogue among stakeholders should 
take place until enlightened agreement is reached* 

Reviewing Resources or Input 
Next, outside evaluafors of an English program may assume that 
human as well as non-human resources are standard and, therefore, 
overlook evaluating them* Indeed, educational reformers tend to pass 
over input lightly while stressing course goals, instruction, and 
outcomes* However, as Charles Cooper states, "students need to read 
and write a lot, and they need response to uheir efforts from 
intelligent, caring teachers 11 (3). Cooper's statement translates in 
part into teachable class sizes and line~items in the budget for 
well-qualified instructors and professors* Even a cursory glance 
gives evidence of the need to evaluate human resources* Although it 
is a major factor in final achievements (Battle 5-10), the students T 
entering preparations vary considerably, even when develop stent a 1 
courses exist (Bowles 2*3)* Class sizes in pre-standard courses of up 
to 100 freshmen instead of 30 to 35 were justified by an experimenter 
when his study found significantly better gains on the standardized 
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Iowa Test of Educational Development CITED) for the larger classes 
(Silver), although it is not clear what the course taught. One 
community college in Texas, Kinneavy reported, had 59 part-time 
instructors, 1 full~time instructor, and no f acu.L. ~y meetings. 
Typically in public universities, according to a national survey 
(Witte, Cherry, and Meyer 46), instructors of freshman English are 54% 
graduate assistants; 19% part-time instructors; 13% full-time, 
non^tenure-track faculty; and 14% tenure-track faculty. Thus it is 
crucial that evaluators plan to review human and non-human resources 
in most English programs. 

An evaluator of resources, or "input," can readily describe the 
students in terms of number in each section, reading level, writing 
level, and ACT-English; instructors in terms of number, rank, 
education, experience, and salary; and other stakeholders in 
appropriate terms. He or she may also assess these resources by 
sending a questionnaire to all persons who expressed needs or rights 
and this time ask what their perceived responsibilities for and 
actual contributions to the program are. How much time do students 
perceive that they owe and do they give to homework? How many hours 
weekly do instructors feel they owe and do they contribute to various 
aspects of the program, such as, commenting on compositions, growing 
professionally, and doing research? What contributions do other 
members of the English department make, such as, serving on relevant 
committees within the department or in the rest of the university to 
communicate needs of the freshman program; supporting t/v\ure and 
promotion; contributing to research; seeking funds for travel and 
development; and assisting new colleagues? Do non-English faculty 
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reinforce what is taught in English by using writing as a tool For 
teaching their own disciplines? Do administrators provide the 
freshman program with funds for full-time faculty, salaries, office 
space, secretarial service, professional development, formative 
evaluation, photocopies, travel to conferences, and released time for 
research? Does the state set aside line-items in the budget for the 
faculty of freshman English programs and tax adequately? These sample 
questions make it clear that every person who has a right to benefit 
from the English program has a corresponding responsibility to 
contribute to the program* 

At this point, professional evaluators might use Provus* 
discrepancy model (245) to search for any gaps between what any group 
expects as goals and what that group gives as resources* For example, 
are students who want to pass but who have inadequate preparation 
doing enough homework and using laboratories? Are instructors giving 
appropriate and healthy amounts of time to students' compositions, 
professional growth, and research? Are other members of the English 
department who expect students to write excellent papers in advanced 
courses giving adequate support to the program's funding needs? Are 
non-English faculty who expect correct academic writing in their 
disciplines using sufficient , appropriate writing to enable students 
to learn the concepts of those disciplines? Are college and 
university administrators supporting their desires foi* grants, 
research, and prestige by providing enough JLine~items in the budget, 
enough sections to permit reasonable class size, and enough credit 
hours for the course? Many additional possible questions arise, such 
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as those in Witte and Faigley's Evaluating College Writing Programs 
(63-65, 75-77). 

The evaluator of an English program who seeks to answer these 
questions about adequacies may, as Dorine said in Tartuf f e (Moli&re 
2.4. Ill), have to "use all manner of means, and all at once." The 
evaluator can first place together each groups' answers to both 
questionnaires* He or she can compare the names of students who were 
recommended to laboratories with the names of those who actually 
attended* He or she can compare the time instructors and professors 
actually gave to reading compositions with the time instructors and 
professors typically give — twenty minutes each paper (Williams 13). 
One can determine whether funding is adequate for teachable class 
sizes by consulting research, such as Glass T meta-analysis on class 
size (42-44); making comparisons with maximum professional standards 
ADE, MLA (2), and NCTE (873); and performing small-scale experiments 
with various class sizes* It may be possible to replicate Weiss and 
Walter 1 © Latin-square procedure to check whether non-English faculty 
members increase their students T learning more when they use writing 
than when they do not use writing* Furthermore, an evaluator can 
compare his or her program 1 s courses and sections with those of 
typical programs, such as, those described by Witte et al* (21-23); 
bring in an outside consultant; or form inter-departmental committees* 

If discrepancies exist, a formal report should be written which 
suggests alternative procedures* Unless goals are sought by means of 
adequate resources, it is highly unlikely that either the process of 
instruction or the students 1 achievements will be satisfactory. 
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Reviewing Process and Product 

Of course, evalustors frequently review the other two elements* 
teaching-learning and achievements* A review of all four elements Is 
Important because, as the Carnegie Task Force reports (ll), all 
Important elements in an educational program may have to be Improved 
for significant change to occur* 

An evalustor who reviews the third element, the processes of 
teaching-learning, can turn tc work already done by CCCC (213-229) 
under Richard Larson's committee* Since the Instruction 1 activities 
that were f^und In the national survey (Wltte et al* 60) to be most 
effective In public universities, that Is, teaching revision (72%) and 
holding conferences (44%) » are highly time consuming, these activities 
may be given special attention. 

An evaluator of the fourth element, students 1 achievements In 
reading and writing, can seek Information from Cooper and Odell's 
Evaluating Writing (1977); publications of the National Testing 
Network In Writing (1981-1986); Davis, Scrlven, and Thomas 1 The 
Evaluation of Composition Instruction; t^tte and Falgley T s Evaluating 
College Writing Programs ; and Edward White's Teaching and Assessing 
Writing* A search for measures of achievement In reading non-flctlve 
literature can be undertaken* Questions concerning all four elements 
can be found In A Checklist and Guide for Reviewing Departments of 
English published by the Association of Departments of English* 

Writing the Concluding Report 
Once the goals, resources, processes, and achievements have been 
reviewed, professional evaluators prepare a report for those who make 
decisions* These experts provide two Insights for assisting freshmen 
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English. The first is the practice of including in the report what 
Rossi and Freeman (267-297) call "cost-effectiveness analyses." Cost- 
effectiveness analyses are calculations of a program's costs in 
dollars but of a program's results in students' achievements* For 
example, an English program for a special group of freshmen might cost 
a great deal but might give the students access to long-term success 
and help the university retain students. 

The second practice, used especially by Scriven (69-70), Stake 
(181), and Stake and Denny (373), is including one last element in the 
report. This element will be explained through two brief an cdotes 
that reveal a characteristic of some faculty members in English 
departments. A researcher. Stalnacker, is said (Diederich 49) to have 
asked a group of English teachers to rank a set of student composi- 
tions into five equal piles ,n order to merit, marking the highest 
pile 4, the next pile 3, and so on down to 0 After completing the 
task, the teachers said that they could find no 4 papers and had 
placed the papers into five piles, ranking them 3, 2, 1, 0, and 00. 
Similarly, on another occasion, when the English committee of an 
accreditation team evaluated the English program of a superb private 
school, the committee members found twenty weaknesses and few 
strengths until their chair urged them to look far positive qualities* 
However, the other committees on the accreditation team assessed the 
English program as the finest in the school, a conclusion supported by 
high SAT-scores and other criteria. 

Therefore, let us speak heartily for the worth of our freshman 
English programs. Freshman English courses are likely to be the most 
challenging courses in a university, the most demanding for students 
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to learn, and the most difficult for faculty to teach. But these 
valuable courses in English are the heart of the university. 

A success story about the benefits of evaluation comes from 
Weaver (13). By consensus, the faculty of Anderson College, Indiana, 
established a standard for writing competency. This standard was 
suited to a diverse group of readers (Hake) because differences among 
raters were compensated for statistically through a computer program* 
Whereas raters of compositions across the university gave fairly 
uniform scores, English had the most extremely high or low reters* 
Results: English now works to articulate goals; English no longer 
reserves A^ f or a paper ready for The Hew Yorker ; each freshman's 
writing is evaluated twice a semester by someone other than the 
English instructor or professor; and only when freshmen can express 
their main ideas and support them with details are they urged to 
notice surface features* 

Evaluation is not easy, but it is necessary to the full existence 
of our freshmen, departments, universities > profession, and nation* 
Professional evaluators ' Insights just described as well as others in 
the ample literature can help make the process of evaluation fair, 
efficient, and thorough* It is especially important to evaluate 
goals, resources, processes of instruction, and achievements, the 
resources being most in need of review* These few insights culled from 
the rest show tha_ professional evaluetors can help us support the 
English programs that enlighten our students, those whom Hilton called 
in "lycidas" (1C7) our "dearest pledgets]*" 
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